Mobility Unified Reporting System Overview


Mobility Unified Reporting System Overview
 
 
This chapter provides an overview of the Mobility Unified Reporting (MUR) application.
This chapter describes the following topics:
 
Introduction
The Cisco Mobility Unified Reporting (MUR) system is a Web-based application providing a unified reporting interface for diverse data from Cisco Systems In-line service and storage applications.
 
The MUR application enables:
This release of MUR only supports generating HTML-based historical canned reports displaying data in graphical—graphs/charts—and tabular formats. Reports for ad-hoc periods are not supported. For information on the various reports supported, see the Report Types section.
The MUR application is available for report generation only when you install the software application on to your local server. For information on the server recommendations, refer to MUR System Requirements section in this guide. For information on how to install the MUR application, refer to Managing MUR Installation chapter in this guide.
The MUR application provides comprehensive and consistent set of statistics and customized reports, report scheduling and distribution from ASR 5000 system i.e. the chassis / in-line service product. For example, a subscriber's Quality of Experience, top 10 sites visited, top 10 users, and so on.
The MUR application provides reporting capability for Content Filtering (CF) data, bulk statistics, Key Performance Indicators (KPIs), EDRs data from in-line service and storage applications. The MUR application facilitates and enhances the operators’ ability to simply and easily determine the health and usage of the network.
Important: In RHEL-based deployment of MUR, L-ESS is NOT required as the ASR 5K Enhanced Charging Services (ECS) module can be configured to push the xDRs directly to the MUR reporting server. Push from ASR 5K is the Cisco recommended deployment model. Currently L-ESS is supported only on Solaris platforms. For information on the L-ESS installation instructions, refer to the Cisco ASR 5000 Series ESS Installation and Administration Guide. Existing deployments where L-ESS is installed, to pull EDRs from ASR 5K, may continue with their deployment model in the 12.0 version of MUR Software Release and later.
For information on obtaining and installing the license, see Cisco ASR 5000 Series System Administration Guide and Cisco ASR 5000 Series Enhanced Charging Services Administration Guide. For information on configuring the ECS module, see Configuring Chassis for MUR chapter in this guide.
MUR receives the following types of EDRs for report processing:
To reduce disk space and improve performance, MUR limits the bucket distribution for EDR data to ONLY last 2 days in case a EDR is spanning across more than 2 days or so.
For example, if the following EDR is received:
#sn-start-time,sn-end-time,radius-calling-station-id,ip-subscriber-ip-address,sn-subscriber-port,ip-server-ip-address,sn-server-port,sn-app-protocol,p2p-protocol,traffic-type,voip-duration,sn-volume-amt-ip-bytes-uplink,sn-volume-amt-ip-bytes-downlink,sn-volume-amt-ip-pkts-uplink,sn-volume-amt-ip-pkts-downlink,bearer-3gpp rat-type,radius-called-station-id,bearer-3gpp imei,ip-protocol,bearer-3gpp sgsn-address,sn-flow-start-time,sn-flow-end-time 1275330600,1275334200,9689944191,19.19.1.1,35111,1.1.1.1,21,8,,,0,52428800,1048576,100,200,1,apn.org1,35302703-090362-52,6,1.1.1.3,1275330600,1275334200
MUR determines the difference between the starttime and endtime attributes and limits the bucket distribution as shown here.
#starttime,endtime,protocol,rxbytes,txbytes 2011/02/26 10:00:00,2011/02/28 10:00:00,HTTP,100MB,100MB
Important: The bucket distribution calculation will remain intact i.e. the volume will be distributed equally among all the half-hour’s buckets that fall in the starttime and endtime.
Important: The MUR receives the data in terms of EDRs which are generated based on the flow. As the EDRs are flow-based and the bulkstats is a real-time data, the volumes reported in the EDR are different from the volumes reported by bulkstats.
For more information on using the MUR application to generate reports, see the Cisco Mobility Unified Reporting System Online Help documentation.
 
Report Types
The MUR application supports generation of canned statistical reports that can be used to analyze network performance, and decide the policies for users, and identify the customer trends, network usage patterns, network categorization, etc. The reports can be per gateway, or multiple gateways (region), or for the overall network. The reports can be generated for the usage of different entities such as gateway, content type, etc on an hourly, daily, weekly, or monthly basis.
The typical canned reports that are supported for the MUR application include:
The static report layout comprises the following sections:
On the interactive layout the user can set a series of preferences in a specific manner. The user has the flexibility to change the type of chart from Bar to Pie (supported output types depend on the selected report). Changing the preferences like the chart type or report parameters will cause the report to refresh in the same window.
The interactive chart layout provides the following list of features:
The MUR application provides the following reports:
MUR supports traffic type detection for P2P protocols such as Skype, Gtalk, MSN, Yahoo, and Oscar with the use of “traffic-type” attribute present in the EDR fields. Based on the value of this EDR attribute, the data will be classified to respective protocols.
The usage traffic is expressed in terms of megabytes (MB) or Megabits per second (Mbps) and percentage (%). The traffic can also be in gigabytes (GB) / kilobytes (KB) / bytes depending on the magnitude.
Important: Active Flow Count report for current date will not be available because daily tables used to fetch this report are generated only at the end of the day. Also when the user selects a date range, for example, 10/1/2011 to 10/5/2011 where 10/5/2011 is the current date, then the report will be shown for the period 10/1/2011 to 10/4/2011 i.e. up till 10/4/2011.
Important: Unique Subscriber Hits report can be generated ONLY for a single date/week/month and not for any date-range. Also, note that the time selection is also disabled for this report.
Typically, this report provides the total number of times a subscriber is using a specific protocol. These reports are displayed for all configured gateways.
Important: Unique Subscriber Hits report for current date will always be available on the subsequent date because unique subscribers hits calculation will be performed at the end of the day.
Important: This report is not available for a multiple date range selection.
After identifying the total amount of transferred data per subscriber, and identifying the top users, to understand the protocol and services breakdown for each subscriber, this report allows listing the different applications used by the top 10/100/1000 subscribers based on the selection of top subscriber per day/week/month.
Important: This report is also available per week or month.
Offline Subscriber Report: The MUR aids in searching individual subscribers’ data based on IMSI/MSISDN, and generates a subscriber-specific report showing the list of URLs visited by the subscriber, and other details like QoS, usage traffic, aggregate application/protocol breakdown, etc for the specified time period. MUR mainly supports this search functionality to track a subscriber or a set of subscribers for lawful intercept.
To use this Offline Reporting feature seamlessly, you must configure the EDR Filename Format appropriately through the Gateway configuration from ADMIN tab, and organize the archive directory date-wise. For information on how to manage the archive directory, see the Managing Archive Directory section in the MUR Administration and Management chapter of this guide.
The offline subscriber report provides information on the following subscriber’s attributes:
Offline Top N Subscribers Report: MUR also facilitates to generate an offline report that covers the % of volume used by top n% subscribers. This report provides information on the absolute number of subscribers and the list of MSISDNs to facilitate correlation with the provisioning data. In this release, this ad-hoc report is available per APN group, Device Group, Location Group, and Service Profile.
Through this custom TopN reporting feature, it is possible to monitor and report the video traffic usage as and when needed. This report is mainly required to identify TopN hosts for video traffic and also to determine the biggest sources of video traffic, which drives the network load at a greater extent.
HTTP content type will be used to identify the video traffic. Ideally video traffic should be derived from flow-EDRs. Since the video usage monitoring report is generated based on HTTP content type, only HTTP traffic will be counted.
For more information on these features, see the Cisco Mobility Unified Reporting System Online Help documentation.
In this release, MUR supports generating daily, weekly and monthly summary details and busy hour traffic usage details for the following report categories:
Important: Release 12.2 onwards, users with only administrative privileges can decrypt the subscriber’s MSISDN to make it appear in the clear text format in the weekly reports.
MUR has the capability to report the following details per protocol:
MUR supports additional information breakdown by network characteristics. These include Application Category, Protocol Groups, IP Protocol, Device Group, RAT (Radio Access Type i.e 2G vs 3G), APN (Access Point Name), SGSN group, Service Profile, Roaming Partner, and Location Group. During its development, a device may have several TAC codes and there may be a need to report devices by broader device type such as "Blackberry" or "Smartphone". Device groups allow the operator to combine a range of TACs into a single named group for reporting purposes.
Busy Hour Reporting: Busy Hour (BH) reporting is mainly useful for the users to monitor different traffic flows in their network during the busy hour. BH indicates the sliding 60-minute period during which occurs the maximum total traffic load in a given 24-hour period.
Please note the following key points:
DSL Reports: The current release of MUR provides the following details for Digital Subscriber Line (DSL) traffic reports:
For information on additional reports supported through DPI, see the Cisco Mobility Unified Reporting System Online Help documentation.
The CF-RE report provides the summary of traffic over CF categories, CF actions, and CF ratings. The CF actions that can be taken on the URL are as follows:
The CF ratings can be one of the following:
The CF-RE report also provides the list of top N subscribers and URLs based on their unique subscriber’s hit count and total usage.
Typically, MUR supports the following categories of HTTP reports:
The top N referrers’ report provides details of the total hit count for top N referrers and their sub-domain wise traffic distribution.
Important: In the distributed model of MUR, the data received from RDP is populated and TopN referrer report is available only at NOC level.
Important: It is mandatory to configure http-url and http-referer fields in the EDR records for top N HTTP referrers report generation.
The report on top N unknown ports can be viewed through the link Edr unknown port infos under the System menu.
Important: Make sure that you configure the bulkstats schemas through the GUI to generate bulkstats reports for any of the available gateways. For more information on schema configuration, refer to the Configuring Bulkstats Schemas Using GUI section in this guide and also Cisco Mobility Unified Reporting System Online Help documentation.
The bulkstat data is sent from the gateway to the MUR server with GMT (UTC Time stamps). The bulkstat file processing is triggered by the MUR scheduler engine. The scheduler processes the bulkstat files line by line for each gateway, and gets the schema, timestamp, and key index. If the index does not exist, the parser creates index and inserts data into bulkstats data table. Once the processing is complete, this data file is moved to the archive directory. Summarization must happen as the user moves from gateway to higher levels.
Important: For Bulkstat, there is no support for distributed model and all the bulkstat input files will be parsed by master MUR system only.
MUR supports generation of busy hour reports, top N Min/Max reports, performance aggregation reports i.e. daily, weekly and monthly summary reports.
Please keep the following key points in mind for bulkstats reporting:
Important: Please note that the Bulkstats and KPI reports are displayed based on the gateway’s time zone.
Important: Please note that the subscriber’s private data like Mobile Station Integrated Services Digital Network (MSISDN) will appear encrypted in all the subscribers reporting. Users with administrative privilege can only decrypt the MSISDNs using a shell script utility. For information on how to use this script, refer to the MUR Administration and Management chapter in this guide. The MSISDN decryption can also be accomplished through Admin > Users menu in the GUI. For decryption through the GUI, see the Cisco Mobility Unified Reporting System Online Help documentation.
Important: Please note that the availability of any report is typically based on the date/date range configurations and purging interval. If you are trying to view a report beyond the configured purging interval, MUR system will display an error message indicating that the report is unavailable.
For more information on each of these reports, see the Cisco Mobility Unified Reporting System Online Help documentation.
 
Exporting Reports to Other File Formats
The MUR application supports exporting reports to the following file formats:
Microsoft Excel format: To export a report to Microsoft Excel format, use the get_excel_report script in the CLI. For more information about this script, refer to the Generating Reports in Excel Format section in the MUR Administration and Management chapter of this guide.
Exporting of reports to Excel format is also possible through the GUI by clicking the excel icon present in the tabular view of each of the reports under HOME and DPI tabs.
PDF format: To export a report to PDF format, in the HOME and DPI tabs of the MUR GUI, click the PDF button. The PDF file is displayed in a new window and can be saved for future reference.
If there is no data available for a report, the PDF button is disabled.
For more information, see the Cisco Mobility Unified Reporting System Online Help documentation.
 
MUR Architecture
The MUR solution consists of two components — a server and a GUI client. The following figure shows a typical organization of the MUR solution.
 
Internal Architecture of MUR
The server components include:
MUR uses pgbouncer utility for postgres connection pooling. This utility gets started/stopped with Postgres Server.
The generators archive the files once they are parsed. In archival, the files are zipped and placed in the configured location.
MUR Parser Server: This will be running as daemons, and it will be spawned at the time of serv start command. Parser server will keep running in background and will perform the parsing activity for all gateways.
The following is a sample output of the serv status command:
---------------------------------------------------
--------------- MUR Process Status ------------
 PID            Process                  Status
---------------------------------------------------
4245          Process Monitor           Running
4256          Scheduling server        Running
4267          Postgres Server         Running
4289          Apache Server           Running
3249          Notif Server           Running
3243          Parser Server           Running
2430          Cache Server           Running
---------------------------------------------------
The following describes the sequential steps associated with the functioning of RPC parser daemons.
1.
If say, Flow-EDR reporting is enabled for GW1, RPC Parser daemon will check the Process Count configured for Flow-EDR under System menu.
2.
 
3.
4.
5.
6.
7.
8.
Some of the components at the client side include Django and Mod_python.
 
Distributed Architecture of MUR
MUR supports the distributed model to allow the deployment which enables network wide view or work load balancing. Newly introduced component, Remote Data Processor (RDP), plays the role of pre-processing the input files from gateways. One or more RDPs, installed separately on remote machines can be registered to a master MUR and one RDP can process files from one or more gateways.
RDP periodically sends the intermediate data to registered master MUR. The role of MUR in such deployments is mostly for report generation, report viewing, RDP management and optionally data processing.
Important: RDP installation and registration is required only for network wide deployments. For standalone installation no RDP is required. For information on how to install the RDP, refer to the Managing MUR Installation chapter of this guide.
Important: Make sure that you first install the master MUR system and then proceed with the RDP installation. Also, note that the RDP and MUR must be installed, upgraded, and uninstalled separately.
Important: Before registering RDP with the master MUR, ensure that the RDP is installed and running.
Important: The RDP management like configuration and removal is possible from MUR GUI only. For information on managing the RDPs, refer to the Cisco Mobility Unified Reporting System Online Help documentation.
Important: For Bulkstat, there is no support for distributed model and all the bulkstat input files will be parsed by master MUR only.
The following figure illustrates the distributed architecture of MUR.
 
Distributed Architecture of MUR
 
How RDP works with MUR
This section describes how the RDP works with the MUR application.
The RDP parses the raw data or EDR files from one or more GGSNs and populates the database for required reports. The RDP pre-processes the data and then periodically forwards them to the master MUR through SFTP for report generation.
Important: If the distributed model of MUR is used, then the SFTP user name and password should be the same as the MUR Administrator user’s login name and password provided during installation. For information on configuring SFTP details, see the Cisco Mobility Unified Reporting System Online Help documentation.
Each of the RDP and MUR will be assigned a unique ID during installation and will be used for identification of each RDP along with its gateway and data.
 
MUR with RDPs in Distributed Model
In 12.0 and earlier releases, each of the registered RDPs will form a new region. RDP region can be a child of the root of the MUR (NOC) or can be the child of another region. The gateways associated with a RDP will always be the children of RDP region.
Release 12.2 onwards, users can create individual regions and add RDPs to the regions. All the gateways must be associated with RDP(s) or NOC and not to a region directly.
Important: Only single MUR can communicate with an RDP simultaneously.
 
Region-based Reporting
In 12.0 and earlier releases, RDP was considered as a region. So, all reports were based on RDP. Whenever an RDP is configured, internally MUR used to create corresponding region for the same. However, with the introduction and need of scalable MUR, one gateway's files will be processed by two or multiple RDPs. In that case, RDP does not stand as a region. So, reports will be required across all the RDPs under one specific region. Particularly, when there are multiple such regions where each region has more than one RDPs, this feature becomes more important. A different case for the requirement of this feature is a region where there are multiple gateways and they are processed by different RDPs. In that case, per RDP based reports will not make sense, rather, region based reports will be required.
Important: In the gateway tree in DPI, HTTP, CF and Bulkstats tab, the pseudo gateway is NOT shown. This is because, there are no specific reports to the gateway, it is just a pseudo to original gateway and all the data is coming from the original gateway only.
 
MUR Deployment
The following figure illustrates how the MUR reporting server interacts with the gateways and generates the reports.
 
End-to-end Component Mapping
The chassis / gateway supports on board Hard Disk Drive (HDD) for extended storage of the xDR files such as EDR, UDR, CDR, and NBR. If the HDD is configured, then the gateway pushes the files to an external entity like External Storage Server (ESS) for short-term storage. In case of no HDD support on the gateway, the Local, short-term External Storage Server (L-ESS) has the capability of pulling the files from gateways via SFTP, and send it for report processing. For more information on L-ESS, refer to the ESS Installation and Administration Guide.
The MUR server collects the EDRs, and bulkstats from gateways or L-ESS server, and processes the incoming data files and presents reports on Web-based GUI. The MUR application can generate reports in Excel, CSV, and PDF formats, and present them to users on a request basis.
Important: L-ESS is NOT required as the ASR5K EDR module can be configured to push the xDRs directly to the MUR reporting server. Push from ASR 5K is the Cisco recommended deployment model. Currently, L-ESS is supported only on Solaris platforms. For information on the L-ESS installation instructions, refer to the Cisco ASR 5000 Series ESS Installation and Administration Guide. Existing deployments where L-ESS is installed, to pull EDRs from ASR 5K, may continue with their deployment model in the 12.0 version of MUR Software Release and later.
For information on how to configure the chassis to push the xDRs, refer to the Configuring Chassis for MUR chapter in this guide.
 
Licensing
For information on licensing requirements for the MUR application, please contact your local sales representative.
 
MUR System Requirements
This section identifies the minimum system requirements that are required for the deployment of MUR at the operator’s premises.
Important: The hardware required for MUR may vary depending on incoming EDR generation, subscriber count, and number of gateways.
 
Server Recommendations for Use in Solaris Environment
This section identifies the minimum system requirements recommended when installing the MUR application in Solaris environment.
NEBS Requirements:
The following are the server specifications for MUR when an additional external storage is required:
Non-NEBS Requirements:
The following are the server specifications with only the internal storage used:
Important: It is strongly recommended to update the Operating System with the latest security patches.
Important: The number of disks recommended is purely based on the throughput of the network and data retention configuration. Please contact Cisco Advanced Service Team for data sizing.
ZFS Pooling Recommendations:
This section provides information on the recommendations for ZFS pooling.
Important: ZFS pool shall NOT be created with RAID-Z since ZFS does not allow attaching an additional disk to an existing RAID-Z pool. Hence, this freezes the chances of data scaling.
 
Server Recommendations for Use in RHEL Environment
This section identifies the requirements of server recommended when installing the MUR application in RHEL environment.
Important: The number of disks recommended is purely based on the throughput of the network and data retention configuration. Please contact Cisco Advanced Service Team for data sizing.
For information related to OS installation, refer to the Cisco MITG RHEL OS v5.5 Application Note.
Important: The Cisco MITG RHEL v5.5 OS is a custom image that contains only those software packages required to support compatible Cisco MITG external software applications. Users must not install any other applications on servers running the Cisco MITG v5.5 OS. For detailed software compatibility information, refer to the Cisco MITG RHEL v5.5 OS Application Note.
Important: ZFS Pooling recommendations are applicable ONLY for Solaris hardware.
 
Storage RAID recommendation for MUR Application
CISCO UCS machine supports MegaRAID controller. This allows configuring the UCS hard disks into hardware RAID arrays (disk groups). The MegaRAID controller provides the BIOS utility for configuring the RAID.
The RAID recommendations for MUR are as follows:
For information on configuring the RAID arrays using MegaRAID BIOS, refer to the Configuring Cisco UCS Servers for MUR System Application Note.
 
Storage Recommendation for MUR Application
This section provides the storage recommendations needed for the MUR application.
Two RAID arrays: RAID-0 for MUR application and RAID-5 for database (Postgres data directory).
LVM: Separate physical volume and volume groups for the three RAID array disk groups.
XFS file-system: block size 4KB, s-unit in terms of RAID stripe size (256KB) and s-width in terms of span of disks in the RAID array.
For information on how to partition storage disk and configure XFS file system, refer to the Configuring Cisco UCS Servers for MUR System Application Note.
 
MUR Ports
This section provides information on various ports and their corresponding port numbers used by the MUR application.
Various ports are used by the MUR for both client-server communication and communication with ASR 5000 system. If firewalls are used on these interfaces, these ports need to be opened.
The following table lists the ports that are used by MUR.
Default Port Utilization
Important: When firewall is used, Apache is the only port that should be kept opened.
Typically, MUR starts all its related services with non-root (i.e. muradmin) privileges.
 
Firewall Settings
When MUR is running on RHEL platform, Firewall is ON by default. In that case, user will NOT be able to get access to MUR GUI. The Firewall MUST be disabled with the following commands:
service iptables save
service iptables stop
chkconfig iptables off
 
Using Apache Port
This section provides information on how to configure the Apache port to use in conjunction with the MUR reporting server.
 
Using Apache in Solaris
In case the user wants to configure Apache port as 80 (i.e. < 1024), it is necessary to run the following command as root user so that muradmin can start the services on ports < 1024.
usermod -K defaultpriv=basic,net_privaddr <mur admin user>
 
Using Apache in RHEL
Important: Make sure that you disable Firewall before using the Apache port in the RHEL environment.
RHEL does not allow port 80 to be used by non-root users. However, Apache Web server requests made on port 80 can be redirected to a port >1024 defined by the operator, with the following two commands:
iptables -t nat -A PREROUTING -p tcp --dport 80 -i eth0 -j REDIRECT --to-port <user defined port> 1024>
iptables -t nat -AOUTPUT -p tcp -d 127.0.0.1 --dport 80 -j REDIRECT --to-port <user defined port> 1024>
For example, to redirect requests made on port 80 to port 8080:
iptables -t nat -A PREROUTING -p tcp --dport 80 -i eth0 -j REDIRECT --to-port 8080
iptables -t nat -AOUTPUT -p tcp -d 127.0.0.1 --dport 80 -j REDIRECT --to-port 8080
Once this is done, user will be able to access the MUR GUI directly, without specifying the port in the Web browser URL http://<serveripaddress>.
 
 

Cisco Systems Inc.
Tel: 408-526-4000
Fax: 408-527-0883